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1 Abstract 

One of the best examples of traditional analysis of evolutionary stable strate- 
gies (ESS) is provided by the so called Dove-Hawk model. In this paper we 
present several enhancements to the model aimed at describing the evolution 
of cooperative behavior. In addition to Doves and Hawks we introduce sev- 
eral groups of Cooperators, who act as Doves within their own group, but 
as Hawks outside it. This allows to study how cooperating groups may grow 
and achieve stability within a nonrepeating evolutionary games framework. 
Depending on initial conditions, the final stable population may have one, 
all-encompassing Cooperator population or several competing cliques. Af- 
ter taking into account that Cooperators bear costs necessary to recognize 
members of one's own group it is possible to see populations where Cooper- 
ators eventually lose against Hawks or populations where several cliques of 
Cooperators coexist with a Hawk population. 



2 Theoretical background 

The classical works by Maynard Smith[l|,0| introducing and popularizing the 
evolutionary game theory and the concept of Evolutionary Stable Strategy 
have resulted in widespread interest in such modeling approach by social 
scientists, biologists and economists. 

*e-mail address: pawelsob@poczta.onet.pl 
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Within the evolutionary games theory one describes a population con- 
sisting of N TO t 'players', who interact among themselves. The form of the 
interaction is simplified to 'contests', which are single events during which 
two players decide how to share among themselves some valuable resource. 
For simplicity it is usually assumed that the resource value is constant (de- 
noted by V). For 'peacefully' resolved contests the value is divided among 
the participant players (equally or unequally). In situations where contests 
develops into a fight over the resource, players participating in the fight may 
suffer injuries, expend unnecessary resources etc., which is modeled by in- 
cluding a penalty cost —h (harm). For simplicity the cost h is assumed to 
be equal for both players participating in a fight. 

There are two main variants of evolutionary games: Nonrepetitive, single 
instance contests, in which participants have no memory of previous en- 
counters and thus cannot adapt their strategies to past behaviour of their 
opponents, and Repetitive Games, where the memory allows complicated 
strategies, individual behavior and learning. 

Within this paper we concentrate on Nonrepetitive Games. Even for such 
simplified situation players may use different strategies. To achieve the status 
of Evolutionary Stable Strategy, specific behaviour a must either: do better 
against itself than any other variant strategy or, if some mutant strategy 
would do just as well against a as a itself, then a must do better against the 
mutant than the mutant against itself 0|. 

The modeling off the population dynamics is best performed assuming 
that the activities take place within separate rounds or iterations. Such 
iterations may correspond, for example, to yearly cycles or biological genera- 
tions - or may not correspond to any discernible quantization of the modeled 
system, but be a convenient simplification. Within such iterations players 
interact among themselves and compete for resources. Between iterations, 
following the average outcomes (payoffs) for different groups, the composition 
of the total player population is adjusted. After several (many) iterations one 
can determine which of the strategies succeeds and becomes an ESS. 

The best known of such models is the Hawk-Dove model of Maynard 
Smith and Price|l|. The population consists of two groups: Doves (D) and 
Hawks (H). These groups are best characterized by their behavior in con- 
tests. When a Dove meets a Dove, they resolve the contest peacefully, with- 
out a fight. The value V is then, statistically, split in half between the 
contestants. Thus the average outcome (payoff) for a player in a D-D match 
is V/2. When a Dove meets a Hawk, the latter 'bullies' the Dove and gets 
the whole value of V, leaving the Dove empty handed. On the other hand, 
when two Hawks meet a fight ensues and the average payoff is V/2 — h. For 
V/2 — h > (if only 'pure' strategies are allowed) the Hawk becomes an ESS, 
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by virtue of winning the contests with the Doves. 

The Hawks and Doves model serves as perfect example that an evolution- 
ary stable strategy is not necessarily the one that returns thee best results 
for the participating players. In a pure Dove population the average pay- 
off of V/2 is greater than in Hawk population. Nevertheless, because the 
outcome of inter-'species' encounters are always favorable to Hawks, even 
a small Hawk population would invade a Dove society and have evolution- 
ary advantage. In a Dove population invaded by Hawks, the average payoff 
diminishes as the proportion of Hawks grows. 

The aim of this paper is to describe some consequences of the extension 
of the Hawk-Dove model within a nonrepetitive paradigm. The extension 
is based upon an introduction of a new strategy, called here 'Cooperator'. 
A Cooperator behaves like a Dove — when it competes with a member of 
its own group. But when a Cooperator encounters a Hawk or a Dove it 
behaves as a Hawk. To obtain more general results we introduce several 
separate Cooperator groups, denoted by Cj, which act friendly within their 
own group, but competitively outside it. In the simplest case, with just one 
Cooperator group, the dynamics of the system is trivial. The Cooperator 
payoff is always better than the payoff for a Hawk. Cooperators could easily 
invade a Dove society. Thus single group Cooperator strategy is an ESS. 

It is worth noting, that although we play the game without 'memory', to 
allow the Cooperator-like behavior some sort of recognition of group mem- 
bership is necessary. This can be achieved through some labeling, pre-contest 
behaviour etc 1 . 

As we are interested in exploring further further than the single Coop- 
erator group, that is into the formation and stability of populations with 
several competitive Cooperator groups Cj. Although in general one expects 
Cooperators to fare better than Hawks and Doves, it is interesting to see in 
what conditions stable population compositions involve just one dominant 
clique and when more cliques form and persist. We have enhanced the range 
of their possible behaviours by including the possibility of transfer of Co- 
operators from different groups (recruitment). In particular, we propose a 
model in which during an encounter between two cooperators from different 
groups first check possibility of recruitment (that is one of the players may 
consider joining the other's group 'on the spot') — and then both players be- 
have friendly in the encounter. Only when recruitment fails the two players 
resort to competitive strategy. In general, inclusion of recruitment compli- 
cates the model, but allows modeling of phenomena such as social group 

1 As always where labeling is present, it opens the field for mimicry or cheating behavior. 
Effects of such strategies will be analysed elsewhere 
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dynamics. One should remark, that such recruitment is possible when Co- 
operators not only invest in some means of recognizing agents from various 
groups (including one's own), but also have some means of establishing the 
reasons for joining (such as results in previous iteration, or group size) during 
an encounter. 

To give our model more precise form, let's define as number of Doves, 
Njj as number of Hawks, N Ci as number of Cooperators of cooperator group 
Cj. The number of Cooperator groups is denoted by Nqg- We assume that 
the population size is constant, Ntot 

N TO t = N d + N h + Y / N Ci (1) 

i 

Additionally we define Nq = J2i Nd ■ 

For a given iteration we introduce average payoffs for each of the groups, 
Pd, Ph and Pq. Assuming that simulation round consists of contests of all 
players among themselves we easily derive: 

Pd = «2 

l\ T OT 

N D V + (Nh - l)(V/2 -h) + N c (V/2 - h) 
h = r T W 



c. 



Ntot 

N D V + (N H )(V/2 -h) + (N Ci - l)V 



N 



TOT 



+ g^j ( Pj ^N Cj V/2 + (l- Pj ^)N Cj (V/2 - h)) 

Ntot 

Within equation^Jthe probability of player from group j joining the group 
of just encountered player % is denoted by Pj^i. Such a probability may, of 
course be modelled in many ways. For the purposes of this paper we use the 
following assumptions: 

Pj^i = for Sji < Tj = tP Cj 

Pj^i = f(Sji) for > Tj = tP Cj (5) 

where Sji is the difference in payoffs for Cooperator group i and group j in 
the previous iteration: 

S 3l = P Cl - P Cj (6) 

The presented model reflects two aspects of the recruitment: a player 
would agree to join a group with better results than his own (Sji > 0) and 
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it would join only if the perceived benefits (based on previous round results) 
exceeded certain threshold Tj. We have set a simple model of the threshold 
being a fixed proportion (t) of the results achieved by player's own group in 
previous round. 

When the threshold is exceeded, probability of recruitment becomes nonzero. 
Different theoretical approaches may use different forms describing the re- 
cruitment probability. We have decided to model the probability by relatively 
simple form of function /: 

/(<y = 6 fi^ (7) 

0j t -h lj 

With this choice probability is small for payoff differences just above the 
threshold, while they grow to ~ 1 when the payoff difference is considerable 
(compared to the threshold for each group), see Figure 1. 

So far the 'intra-iteration' population dynamics were discussed. When the 
phase describing agents encounters is finished, then, according to the values of 
payoffs for each 'species' and group we model the 'inter-iteration' dynamics. 
Here again, flexibility of choice for the mechanisms is considerable. Within 
this work we use a relatively straightforward form of updates to populations. 

The payoff for each group P a (where a stands for D, H, or Cj) is compared 
to the average payoff Pav for the whole population. Normalized deviation 
for group a is defined as: 

D a = PJPav ~ 1. (8) 

We define then 

D = JZdL (9) 

y a 

<P a = DJD. (10) 
The new size of group N' a is then modelled as follows: 

K = N a if \D a \<T G (11) 

where M is a normalizing factor ensuring that the total population remains 
constant and N a is the size of the group after the recruitment phase. S is an 
artificial damping factor used to decrease the scale of inter-iteration changes 
and smooth out simulations. Effect of S is marginal - it affects mostly the 
number of simulations until stability is reached, but not the final outcome. 
The threshold Tq is introduced to model the situation in which relatively 
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small (i.e. smaller than T G ) differences in payoffs should not have significant 
effect on population dynamics. The reasoning behind such a threshold is 
based on observation that in 'real life' situations, factors other than direct 
competitiveness influence the survival and changes in population and may 
smooth out the payoff differences if these are sufficiently small. 



3 Recognition costs 

One of the most interesting simple enhancements is an inclusion of 'label- 
ing cost', that is the cost borne by Cooperators to provide reliable means 
of recognition and determination of group membership. The costs may have 
different origin and form, depending on the modelled system (such as behav- 
ioral costs, costs of group uniforms etc.). 

We propose here a simple approach in which the labeling cost / is equal 
for all Cooperators and is applied to all contests, before the contest is re- 
solved. This corresponds to Cooperators investing in specific behavior or a 
recognizable 'uniform' displayed at all meetings with the purpose of showing 
the group membership. In such case the payoff for Cooperators in all contests 
is diminished by I. Thus the expected payoff for a Cooperator from group Cj 
becomes: 



N D (V-l) + (N H )( V/2 -h-l) + (N Cl -1)(V- I) 

n,d - 



Ntot 

(pj-^Nc, (V/2 -/) + (!- Pj ^)N Cj (V/2 -h-lj) 

+ AT ( > 

Obviously for I > h Cooperators are at distinct disadvantage to Hawks 
and do not have any chance to become an ESS. On the other hand, for the 
opposite situation (I < h) we have quite interesting situations. Lets first 
concentrate on a simple case with just one group of Cooperators, competing 
with Hawks and Doves. 

The difference in expected payoff for the Cooperators and Hawks becomes 

Pi,c-Ph = -^—(N D (y-l) + (N c -l)(V/2-l)+N H (y/2-h-l) 
-N D V - N c (V/2 -h)- (N H - l)(V/2 - h)) (13) 

assuming that Nh, No > 1 we can simplify the above expression to 

Pi,c -Ph= -t^— (N c (h -I)- (N H + N D )l) (14) 



6 



As the Doves are at disadvantage and after sufficiently long evolution would 
die out, we have a simple condition 



. The interpretation of the above condition is that a small cooperator group 
would not be able to invade a well established Hawk society. Thus the ratio 
l/(h — I) determines the course of evolution through initial conditions (as 
discussed in the next section). 

4 Results of simulations 

As the number of possible combinations of the input parameters is enormous, 
we have selected several examples of simulation results, showing typical be- 
havior. These results are presented in the following Tables 1 and 2 and 
Figures 2-9. In the presented simulation examples we have started from 
relatively large Dove population (92% of the total population) and small ad- 
mixtures of Hawks and Cooperators. The initial sizes od the Cooperator 
groups were assigned randomly. Payoff parameters were assigned in a way to 
reflect non prohibitive fighting costs and labeling costs (V > h > I). 

A few general trends can be seen from the presented simulation results. 
First, for all cases except P, population of Doves decreases to zero. This is an 
obvious result of the disadvantages that Doves have when faced with more 
belligerent groups. In the presented simulation in which Doves do not die out, 
we have assumed relatively high value of Tq threshold, equal to 50% of the 
average payoff. For such large value, after some iterations, the populations 
become frozen (see Figures 8 and 9), as the payoff difference is too small to 
affect any changes within our model. 

Second, for simulations with zero labeling constant and 'reasonably' small 
thresholds, Hawks die out as well. Obviously, the ability to resolve contest 
costlessly within a Cooperator group always gives Cooperators a better po- 
sition. 

Interesting behavior may be seen when the labeling cost is nonzero. Then, 
depending on the initial set of control parameters the end result may be, 
in addition to Cooperators, Hawk population may exist. The dynamics of 
the population becomes very dependent not only on the control parameters, 
but also on the initial conditions. For example, simulations K and M have 
the same set of control parameters, differing only in the initial number of 
Hawks and Cooperators. Although in both cases, the respective sizes of the 
populations Nh and Nc are smaller then few percent of the total population 




H 



(15) 
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Initial populations 



Simulation 


A 


B 


c 


D 


E 


F 


G 


H 


Ntot 


10000 


10000 


10000 


10000 


10000 


10000 


10000 


10000 
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9200 


9200 
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9200 
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9200 


N H 


500 


300 


300 


300 


300 
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150 


150 


N CG 
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6 
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Payoff and threshold parameters 
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12 


12 
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0.25 
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5392 
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Table 1: Examples of simulation results for labeling cost / = 0. Columns 
A — H describe different simulation runs. 



size, the final composition is radically different. This illustrates the fact that 
the initial conditions can play significant role in the population dynamics. 

5 Discussion 

5.1 Hawk-Dove-Possessor 

In a recent paper Yee has presented analysis of the evolutionary stability 
of a different example of enhancement of the Hawk-Dove game. The analysis 
was aimed at situations where there are definite resources that can be 'kept' 
between iterations or claimed before contests. Yee described results of a 
strategy called 'Possessor' (P), which is defined as follows: Possessor behaves 
like a Hawk if it is currently the 'owner' of the resource (defending its right to 
is vigorously) and like a Dove if it is not an owner (accepting the 'first come 
first served' mentality). The new strategy, even though conceptually very 
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Initial populations 



Simulation 


I 


J 


K 


L 


M 
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10000 


10000 


10000 


10000 
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N D 
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5654 
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10000 
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1 
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1301 
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92 







796 
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13 

















43 







13 
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Table 2: Examples of simulation results for nonzero labeling cost I. 



simple proves to be effective against Hawks and Doves. Moreover, behavior 
like this is observed both in human societies and in animals. Nevertheless, 
the Possessor strategy, does not lead to cooperation between players, and is 
actually a very individualistic position. 

5.2 Crucial role of thresholds to obtain nontrivial solu- 
tions 

In contrast to the repeated PD games and other studies including learning, 
encounter memory and adaptive strategies of players the model presented in 
this paper does not touch the issue of formation of Cooperator like behaviour 
and Cooperator groups. Our aim was rather to study the factors which might 
influence the dynamics of the populations, especially whether it is possible 
to obtain stable and mixed populations. 

The general tendency shown by simulated populations shows that, in- 
deed, such mixed populations are possible. The key role allowing their for- 
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mation is played by the two thresholds introduced into the model. Without 
those thresholds arbitrarily small differences in payoffs may dominate the 
population evolution. As mentioned before, the commonsense reason for the 
introduction of the thresholds has its roots in the cumulative uncertainty 
of the effects of the factors which may determine the payoff for individual 
player, which are not modelled in our work. Such 'noise' (for example luck 
of individual players in finding necessary resources) would wash out the pay- 
off differences between groups resulting from discussed strategies if they are 
sufficiently small. 

There is an interesting similarity between the presented model and the 
analysis of Frank Q of the need of investment in the display and recogni- 
tion of intentions and threats. Such displays are important in repetitive PD 
games, ensuring cooperative behaviour. In our case, within the framework 
of Nonrepetitive PD games, labeling allows recognition of group member- 
ship and thus of the strategy of the agents in an encounter before the costly 
competition takes place. To recognize the group membership (and thus the 
strategy) the agents must, however, pay 'recognition costs'. Such situation 
leads to stable mixed population, because the benefit for Cooperators at be- 
ing able to recognize Defectors and shirk from meeting with could be balanced 
by the recognition costs. 

The key result of the paper is that a very simple model may lead to the 
establishment of several types of populations — corresponding to situations 
found in biology or sociology. Depending on initial conditions, the final 
stable population may have one, all-encompassing Cooperator population 
or several competing cliques. After taking into account labeling costs it is 
possible to simulate populations where Cooperators eventually lose against 
Hawks or populations where several cliques of Cooperators coexist with a 
Hawk population. 
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6 Figures 
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Figure 2: Example 1 — Evolution of population composition ending in one 
dominant Cooperator group. Parameters used: V — 12, h — 4, 1 — 0, t — 0.1, 
T G = 0.1. 
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Figure 3: Evolution of payoffs for the Example 1 
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Figure 4: Example 2 — Evolution of population composition ending in several 
Cooperator groups. Parameters used: V — 12, h — 4, I — 0, t — 0.3, 
T G = 0.3. 
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Figure 5: Evolution of payoffs for the Example 2 
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Figure 6: Example 3 — Evolution of population composition for simulation 
with nonzero labeling cost ending in several Cooperator groups and Hawk 
population. Parameters used: V — 12, h — 4, I — 1.5, t = 0.3, Tq = 0.3. 
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Figure 7: Evolution of payoffs for the Example 3 
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Figure 8: Example 4 — Evolution of population composition for simulation 
with nonzero labeling cost ending in mixed Dove, Hawk and several Coop- 
erator groups. Due to quite high Tq value of 0.5 after several iterations 
differences in payoffs no longer change thee populations. Parameters used: 
V = 12, h = 4, / = 2, t = 0.5, Tq = 0.5. 
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Figure 9: Evolution of payoffs for the Example 4 
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